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SUMMARY 


Pm  hie  m 

In  recent  years  tliere  has  been  increasing  interest  shown  in  data-adaptive  signal 
processing  tecliniijues.  Ot  special  interest  has  been  a method  now  known  as 
maximum  entropy  methods,  or  MUM  lor  short. 

Wliile  MI  M has  been  sliown  to  be  superior  to  standard  Fourier  techniques  in 
sc'paration  of  very  close  frequencies,  no  delinitive  studies  have  been  published  which 
analy/ed  the  variance  in  the  spectral  estimates  obtained.  Ibis  report  investigates  this 
aspect  ol  MFM  analysis. 


Results 

Maximum  entropy  methods  ol  spectral  analysis  have  been  analyzed  as  to  the 
effects  of  autoregressive  I AR)  process  order  (Ml  signal  to  noise  (S  N I.  dc  offset, 
frequency,  and  initial  phase.  I he  results  show  that  the  location  ot  the  spectral 
peak  (I  P)  is  a function  of  each  of  these  save  dc  offset  In  particular,  and  m contrast 
to  F'ourier  analysis,  the  I P bias  is  a function  of  signal-to-noise  ratio  It  is  also 
heavily  dependent  upon  the  order  of  the  AR  process.  For  the  particular  case  ol 
M = 1().  S,N  = loot).  Fourier  techniques  bias  is  less  in  magnitude  and  leads  MI  M 
bias  by  rr  2 radians.  For  both  MFM  and  Fourier  translorm.  the  ellect  ol  initial  phase 
is  to  shift  the  bias  phase  by  a corresponding  amount.  Power  estimation  requires 
integration  limits  which  are  functions  ol  S'N.  location  ol  adjacent  spectral  peaks, 
and  the  AR  order,  and  it  is  thus  a difficult  parameter  to  estimate. 
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INTROlH'dlON 

M.ixinuini  ciilropy  luctliods  ol  spectral  analysis  (MI  M I arc  currently  in  use  at 
\()S(  aiul  at  other  lacilities.  aiul  some  concern  has  Iven  noted  about  excessive 
variance  in  the  estimation  ot  spectral  (]uantities  using  this  method.  Several  authors 
have  mentioned  such  variance  I Kel.  l-.^l.  and  this  report  presents  an  experimental 
investigation  into  the  problem. 

Since  the  MI  M algorithms  are  by  now  well  established  in  the  literature,  this 
report  does  not  include  the  theoretical  development  ol  the  relationship  between 
entropy  aiul  autoregressive  (ARt  processes.  Relerences  2 and  4 provide  a very  good 
theoretical  discussion,  and  Relerence  .*5  gives  a very  usable,  programmable  algorithm 
as  well  as  an  excellent  discussion  ol  the  devektpmeni  ol  MI  M. 


hXPkRIMI  NT AL  PAR A.MLThRS 

The  lime  domain  data  stuilies  consisted  ol  a cosine  w'ave  at  a lixed  Ireijuency 
and  phase  mixed  with  normally  distributed  random  noise.  I he  Irequencies  studied 
were  arbitrarily  chosen  to  range  between  2 N and  4'N  hert/,  inclusive  Phases  were 
0.0.  nl4.  and  n 2 radians.  I he  record  length  N was  b4  samples,  with  a Nyquist 
Iretpiency  ol  O.*'  11/  I he  sample  mean  was  always  subtracted  trom  the  data  in 
order  to  allow  a better  evaluation  ol  the  etiects  ol  dc  ollset.  .Signal-lo-noise  power 
ratios  (S  Nl  were  0 5.  2.0.  ‘'0  0.  200.0.  and  5000.0.  No  attempt  was  made  to 
predict  a correct  AK  process  order  (see  Ret'.  (i  and  7),  but  the  ellect  ol  order  was 
studied. 


EXPERIMENTAL  RESULTS 

The  basic  inlormation  being  sought  in  litis  study  was  an  estimation  ol  the 
location  of  spectral  peak  ( I.Pl  and  the  variance  of  the  estimates  about  a mean  value 
Of  the  parameters  expected  to  inlluence  LP.  namely  S N.  frequency,  sample  ollset. 
initial  phase,  and  AR  process  order,  the  first  and  the  last  must  be  presented 
together 
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Order  ot  the  Autoregressive  Pr«»eess 


I he  inituil  et)neern  was  tlie  elieel  ol  the  AK  proeess  order  ( M ) on  tlie  speetnim. 
It  has  heen  reported  hotli  tliat  tlie  eorreet  order  is  eriieial  (Ret.  2),  anil  that  little 
impaet  is  notieed  within  eertain  ranges  (Ret  1 ).  Speetral  eomputatioii'  were  made 
tor  each  ot  two  treiiueneies  using  three  dilterent  S N\  ot  2.0.  50.0.  and  5000.  I he 
values  ol  M and  the  ealeulaled  biases  are  tahulated  against  S/\  in  F able  1.  liias  is 
defined  to  he  the  deviation  Irom  the  true  11’.  aseomputed  from  an  infinite,  eonlin- 
uous  wave.  From  Ihe  lafile.  two  faetors  are  immediately  appareni  First,  for  a 
“lundamenlal"  frequeney  la  multiple  of  1 N).  biasefTeets  are  mueh  more  eonsistent 
than  lor  an  "oH"  frequeney.  In  lael.  lor  the  seeotui  frequeney  shown.  Ihe  worsi 
S N ol  2,0  has  Ihe  esiimalor  bias  ehanging  sign  over  an  absolute  range  of  .50.5  x I0~^ 
II/.  while  Ihe  First  frequeney  is  at  least  unilormly  ol  the  same  sign.  Fven  at  an  S N of 
5000,  the  Frequeney  has  a range  oFb..5|  x lO""^  II/.,  while  Ihe  Fundamental  Freiiueney 
has  a range  ot  O.XS  x 10— ^ II/.  I he  seeond  Feature  is  the  variation  For  a given  M at 
either  Frequeney  as  S N is  redueed.  In  some  eases,  as  For  M = 20  For  Frequeney  = 

2/N  II/.  the  hia.s  aetually  inerea.se.s  with  increasing  S/N,  but  the  more  general  condi- 
tion shows  a decrease  in  bias  with  increasing  S/N.  It  should  be  noted  here  that  each 
tabulated  value  ol  bias  represents  only  a single  noise  reali/.ation.  rather  than  an 
ensemble  average,  and  that.  For  a given  M.  the  noise  reali/alion  is  the  same.  Because 
ol  monetary  constraints,  a detailed  statistical  evaluation  ot  the  ell'eet  ol  AR  order 
was  not  possible.  However,  to  con  Firm  the  apparent  variation  caused  by  the  relation- 
ship between  M and  S N.  ten  reali/alions  w'cre  averaged  For  each  ot  Four  Fre()uencies. 
tor  selected  S N ratios.  An  AR  order  ol  lb  was  chosen.  The  results  are  shown  in 
Fig  l it  should  be  noted  that  only  statistically  significant  estimates  are  shown  on 
the  graph.  Fable  I and  Fig.  1 show  that  For  MFM  analysis,  bias  is  a Function  both  of 
signal-to-noise  ratio  and  AR  order.  Fhis  is  not  the  case  For  Fourier  spectral  analysis. 


Bias  Estimation  fur  High  .Signal-to-Nnise  Ratio 

As  a comparison  with  Fourier  techniques  (FT),  the  bias  generated  by  proximity 
to  the  zero  Frequency  axis  was  investigated.  For  Fourier  techniques,  the  l.P  is 
contaminated  by  the  side  lobes  ol  the  negative  Frequency  component.  I he  contami- 
nation. or  bias,  lakes  the  Form  oFa  damped  sinusoid,  decreasing  in  magnitude  as  one 
leaves  zero  Frequency.  Since  it  has  been  shown  that  bias,  in  the  MFM  case,  is  not 
only  a Function  ot  signal  to  noise,  but  is  also  inconsistent,  a high  signal-to-noise  ratio 
ol  5000  was  used  tor  this  study.  Ihe  results  are  shown  in  Fig.  2 tor  M = lb.  As  can 
be  seen.  For  these  parameters  the  MEM  and  I T biases  are  approximately  rr/2  radians 
apart,  and  Ihe  MFM  bias  is  considerably  greater  in  magnitude.  Further,  although  not 
enough  Frequency  points  were  used,  it  appears  that  the  bias  is  itself  biased  toward 
the  positive. 


i 


Initinl  Phase  litTecIs 

1 ijjiirc  2 also  shows  the  elYeet  ot  Iliree  tlilTorenl  initial  pliases  for  holli  Mf-M 
am)  I I lor  two  lre(|ueiK'ies.  As  ean  be  seen,  for  botli  methods,  a elianpe  in  initial 
phase  results  in  an  equal  shilt  in  bias  phase. 


Eflecls  of  Sample  Offset 

.As  noted  earlier,  the  sample  mean  was  removed  Ifom  the  data  in  all  eases  to 
facilitate  an  understandin}i  of  the  elfeets  of  nonzero  mean  in  the  time  domain  data 
.A  constant  “dc”  ofiset  t)f  1.0  was  added  to  time  samples  at  frequencies  2'N  Hz  and 
2..n  N llz.  No  appreciable  effects  were  discernible  in  the  I P for  the  MHM  analysis 
for  either  Irequency. 


Power  Estimation 

In  the  course  of  this  investigation  an  attempt  was  made  to  estimate  the  pt)wer 
in  the  various  fre(]uency  components  in  the  waveform.  I his  necessitates  an  integra- 
tion process  since  the  AR  method  is  basically  an  energy  density  approach.  Unfor- 
tunately. no  valid  results  were  obtained.  In  order  to  estimate  the  energy  (or  power), 
appri)priate  limits  of  integration  are  required.  While  for  very  high  signal-to-noise 
ratios  it  is  possible  to  empirically  determine  relatively  acceptable  limits,  these  limits 
are  functions  of  signal  to  noise,  proximity  to  other  peaks  in  the  spectrum,  and  the 
orrler  of  the  AR  process. 


DISCUSSION 


MEM  analysis  has  Iven  shown  by  several  autlu)rs  to  be  a valuable  tool  in 
estimating  the  frequency  components  ol  a complex  waveform.  By  its  very  nature  of 
being  data-adaptive  it  tends  to  "find"  the  spectral  quantities  rather  more  rapidly 
than  Fourier  techni(|ues  However,  it  has  several  drawbacks  which  might  tend  to 
limit  its  uselulness:  there  is  no  exact  way  to  determine  precisely  the  correct  AR 
order,  the  estimates  are  biased  by  high  signal  to  noise;  it  is  quite  difficult  to  deter- 
mine appropriate  frequency  limits  of  integration  when  attempting  to  estimate 
power  None  of  these  problems  occur  with  Fourier  spectral  analysis. 
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I igurc  I Ml  M c^tlmatc&  of  bias  for  various  S/N  ratios. 
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Ijhlc  I.  rtlcil  «>t  AK  tudof  (M)  on  hias  cstinialion. 


M 

S N 

Hiavl*  10^  Mil/) 

S/N 

Kijs  (X  IO“^Mll/) 

S/N 

I3ijs  )/  lO^MlI/) 

Nominal 

1 fc<|ui-iuy  = 

’/N  003125  11/ 

H 

■4.VH 

50 

7.36 

2 0 

30  02 

10 

5(M»0 

4.83 

50 

7.13 

2.0 

36.55 

12 

MHMI 

4.91 

.50 

6.73 

2.0 

37.15 

M 

5(KK> 

5 02 

50 

6 19 

2.0 

19  98 

IK 

5000 

4,82 

50 

502 

2.0 

16.82 

20 

5(MM» 

4.78 

50 

5 02 

2.0 

2.49 

22 

$imi 

4.38 

.50 

7.05 

2.0 

1.3.41 

24 

SO(M» 

4 42 

SO 

6.81 

20 

6.41 

2f> 

SfMMt 

4 29 

SO 

6.23 

2.0 

4 76 

2H 

so<io 

5.17 

50 

5 .34 

2.0 

16.25 

Nominal  1 rci|ucm  > 

2.37/N  = ().0.37()3I2S  11/ 

8 

50(H) 

-9  21 

50 

-22.17 

2 0 

-.5.50 

10 

5(MK) 

-7.59 

50 

-11.16 

2.0 

866 

12 

5(8)0 

-7.66 

50 

-7.41 

2 0 

11.27 

14 

.5)MM) 

-8.89 

.50 

-15  63 

2.0 

-3.34 

18 

50(K) 

-7.69 

50 

-5.67 

2 0 

7.05 

20 

5(MM) 

-8.03 

50 

-6  84 

2.0 

-7.61 

22 

5(M)0 

-903 

.50 

-13.83 

2.0 

- 1 9 23 

24 

5(MH) 

-10,25 

50 

- 1 3.06 

2.0 

-8.69 

26 

,5(MM) 

-11.35 

50 

-13.41 

2.0 

-17  98 

28 

5(XM) 

-5.04 

50 

-3.40 

2.0 

6.36 

t 

1 
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TABULA  I ION  OI  RI  SULTS 


I ho  lollowin)!  t;ihlo  prosi’iils  ;i  oomparison  tvlwoon  I T and  MI  M tor  various 
S N ralii's  Tor  Ilio  Iroquonoios  indicated  in  l ig.  I and  2.  "L  I OM  ” “MLM 
are.  rospoctivoly . the  standard  deviations  ot  the  l ourier  bias  and  the  MhM  bias 
riie  lu'ading  "no.  ot  o’s”  represents  tlie  number  ot  standard  deviations  of  the  bias 
A value  under  this  heading  which  is  greater  than  about  d may  be  considered 
statistically  signilicant.  ! he  AR  order  is  lb.  and  the  numlxu  ot  realizations.  N.  is  lO. 
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Tahlf  A I Slah^tual  lomparison  of  M and  MI  M bias. 


1 1 Bias  (X  lO'*) 

1 1 1>M<X  10'*) 

No.  of 

Ml  M Bias 

MtM  oM 

No.  of 

S N 

(H/) 

lll/l 

o*s 

(X  t0-*l  (ll/l 

(X  10-<Mll/) 

o's 

1 rcqucmy 

0.0^  1 25  H/  . phase 

0 0 radians* 

0 5 

-0  KK5 

21.772 

0.04 

24  051 

4.4721 

5.31 

:<> 

7.281 

2 498 

2 92 

1 3 ,540 

1.325 

10.22 

5tMl 

10.523 

0.4  55 

23  13 

5.984 

0.191 

31.33 

2(K).0 

10  8.1h 

0.228 

47.53 

5.234 

0 115 

45.51 

5(M)0.0 

11.031 

0.044 

2.50.71 

4 904 

0 024 

204  33 

1 ri-qucncy 

= 0.03220  11/.  phase 

• 0.0  radians* 

0.5 

-5.h64 

5.875 

0% 

12.33 

5.216 

2 .36 

2.0 

1 594 

2.084 

0 765 

-1  686 

2.069 

0.82 

50.0 

3.953 

0.3% 

9.98 

-11.454 

0.482 

23.76 

2(M).0 

4.172 

0 197 

21.18 

-1 1 469 

0 281 

40  82 

50000 

4.3.36 

0.387 

1 1 20 

-1  1 146 

0.207 

53  85 

1 rcqucniy 

0.035 16  11/ . phase 

= 0 0 radians* 

0 5 

-9.313 

4 339 

2.15 

1.568 

6 507 

0.24 

2 0 

-4  820 

1 947 

2.48 

-12  932 

3.352 

3.66 

50.0 

-3  195 

0.374 

8.54 

-19  889 

2.095 

9 49 

2(K»0 

-3.039 

0 1 90 

16.0 

-24  .(XI 7 

2 311 

10.39 

5000.0 

-2  914 

0(MM 

728.5 

-17.644 

2.4  84 

7 10 

i rcqucney 

•0.03711  II/.  phase  = 

' 0.0  radians* 

0.5 

-7.867 

6.548 

1.20 

-2  865 

7.669 

0 37 

2.0 

-9.281 

2.120 

4.38 

-13  804 

4 4 70 

3 10 

50  0 

-8.577 

0 383 

22.4 

-9  564 

1 440 

6 64 

200.0 

-8492 

0 1 89 

44  93 

-9  085 

0 803 

11.31 

5(HK).0 

-8.445 

0 038 

222.20 

-8  203 

0.210 

39(Xi 

1 rcqucnty 

« 0.03906  11/:  phase 

* 0.0  radians* 

05 

-7.258 

6.758 

1.07 

1 405 

8.036 

0 18 

2.0 

-8  899 

2.657 

3 35 

-3  251 

4 820 

0.68 

50  0 

-9  641 

1 393 

6 92 

5.652 

5 017 

113 

200  0 

-9.664 

0 218 

44.33 

6 171 

0 897 

6.88 

S«(K)0 

-9.71  1 

0.0(M 

2427.75 

6 741 

0 223 

.30  23 

• DC'  offwt  of  1 .0  addl'd  to  bias.* 
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Table  A-l  (('ondnuedi 


S/N 

n Bias  (X  mH) 
111/) 

1 TomO<  >0-*) 

(ll/l 

No.  ot 
o's 

MhM  Bias 
(X  10-<l  (11/1 

Ml  M OM 

(X  l(l“*l(ll/l 

No  o 
fl's 

1 rcqui'iK> 

= 0.04  297  H?,  phase 

= 0 0 radians* 

0 5 

11  505 

5.083 

2.26 

22(141 

5 529 

3 99 

2.0 

7.255 

2.6  29 

2.76 

23  020 

3 191 

7 21 

50.0 

4.029 

0.539 

7.48 

30.571 

1 142 

26  77 

2<IOO 

.V662 

0.269 

13.61 

32  693 

0 806 

40  56 

5000  0 

3.380 

0.053 

63.77 

25.724 

0.469 

54  85 

1 Tcqucncy 

- 0 04688  11/,  phase 

= 0.0  radians* 

0 5 

15  71 1 

4 835 

3.25 

18.230 

4 942 

3 69 

20 

10.844 

2 412 

4.50 

13.531 

1.850 

7.31 

50  0 

8.102 

0.498 

16.27 

7.909 

0..307 

25.76 

2(K)  0 

7 805 

0.245 

31  86 

7 279 

0.157 

46.36 

5000.0 

7 563 

0.0.54 

140.06 

6.592 

(1.066 

99  88 

V requrmy 

» 0.031  25  11/.  phase 

n/4  radians* 

5(MK»  0 

2 242 

0 06  3 

35.59 

1.032 

0 (148 

21  5 

1 lequcncy 

* 0.031  25  11/:  phase- 

ir/2  radians* 

5(HX)  0 

-11  953 

0(153 

225.53 

-4  647 

(1(182 

56.67 

1 requcn^y 

• 0 0371 1 11/  ; phase 

= IT  4 radians* 

50(Kt  0 

•2.976 

0.037 

80  43 

1.783 

0. 1 73 

10.31 

frequency  0 0371  1 H/.  phase* 

it/2  radians* 

5(KMM) 

8.220 

0.060 

137. 

12.925 

(1  285 

45  35 

1 roquency 

• 0.03 1 25  H/ . phaw 

* 0.0  radians* 

5(KX)<) 

11  031 

0(144 

250.71 

4 904 

0 025 

196.16 

1 tfqurmy 

• 0 0371 1 11/  . phase 

* (10  radians* 

5(XX),0 

-8  44  5 

0037 

22.59 

-8  203 

0.21(1 

39.06 

• IX’  uff&ct  of  1 .0  added  lo  bias*  • 
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